Activity Modeling with Spatio-temporal Texture Primitives

In this paper, a novel structure is proposed for human activity modeling using time sequential spatio-temporal texture primitives. Gabor filters, which are proven to be robust 2D texture representation tools, are extended to 3D domain to capture spatio-temporal texture features. A well known filtering algorithm and an unsupervised clustering algorithm, the Genetic Chromodynamics, are combined to select salient spatio-temporal features. Each state of activity is represented as action units with its salient spatio-temporal feature set, which are also the symbols of our codebook. To overcome temporal variation between different performances of the same action, a profile Hidden Markov Model is applied with Viterbi Path Counting (ensemble training). Not only the parameters and the structure but also the codebook is learned during training. Proposed structure enables to recognize sub-sequences and does not require end point constrains. This also provides robustness to the missing data and occlusions.